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Executive Summary 


This report provides information about new teachers’ preparation experiences and explores whether 
particular types of experiences are related to teachers’ effectiveness in improving their students’ test 
scores. 


Prior research indicates that teaching effectiveness is the largest in-school factor affecting student 
achievement (Chetty, Friedman, and Rockoff 2014a, 2014b). Research also shows that new teachers are 
less effective than more experienced teachers (Clotfelter, Ladd, and Vigdor 2007) and frequently begin 
their careers in high-poverty schools (Bruno, Rabovsky, and Strunk 2019), where students are in greatest 
need of effective instruction. Improving the preparation that teachers receive is a potential strategy for 
increasing new teachers’ effectiveness and for closing the student achievement gap, which is a primary 
goal of the Every Student Succeeds Act of 2015." 


Understanding whether certain ways of preparing teachers are more effective than others is important, but 
very little is currently known about what teacher preparation emphasizes at a fine-grained level and how 
those experiences are related to teaching effectiveness. Research identifying which preparation 
experiences are related to improved student performance can generate hypotheses for improving teacher 
preparation. 


About the Study 


Through an online survey administered in spring 2015, this study collected information from new 
teachers about their preparation experiences. It also examined the relationship between the frequency of 
these experiences and teachers’ effectiveness in improving student test scores. 


Study Sample 


The study recruited a sample of large districts/states that could provide student-teacher linked 
achievement data for the calculation of teacher value-added. The sample consisted of 242 districts in 18 
states. These were primarily large, urban districts, located in the South, with high levels of students in 
poverty and high proportions of minority students and English learners. 


Within these districts, 3,294 teachers in grades 4 through 6 completed the survey. This large, 
opportunistic teacher sample was similar to a nationally representative sample of novice upper elementary 
teachers from the 2011-12 Schools and Staffing Survey (SASS) in terms of sex, age, and certification 
route (alternative or traditional). Study teachers had participated in the preparation programs of 566 
different providers. 


To examine the relationships between preparation experiences and teaching effectiveness, the study 
collected student test score data for a subset of 2,533 of these teachers.” 


1 Every Student Succeeds Act of 2015. P.L. 114-95 § 114 Stat. 1177 (2015-2016). 


2 Because of considerations of cost and expected numbers of surveyed teachers in each district/state, we limited 


the collection of student-teacher linked achievement data to the 19 largest districts/states. 
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Preparation Experience Measures 


The survey asked study teachers Figure ES.1: Taxonomy of Preparation Experiences for 
about the preparation experiences This Study 
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6. Facilitating extended classroom discussions 

7. Designing and using assessments of student learning 

8, Building students’ higher order thinking skills PROMOTING 

9. Providing feedback that helps student learning ANALYTIC 

10, Building comprehension of academic concepts THINKING SKILLS 
Ni. Effective ELA instruction 

12. Effective math instruction 

13. Effective instruction for English learners 


4 TYPES OF LEARNING OPPORTUNITIES 


COURSEWORK | OBSERVATION | PRACTICE | FEEDBACK 


The teacher survey was developed ‘Each compatency area consisted of five exemplary instructional strategies 

based on prior research that linked 

classroom practices within each competency area with gains in student achievement and in consultation 
with national experts.” It asked the study teachers to rate the extent to which, during their preparation 
program, they had experiences with each of the instructional strategies obtained through each of four 
types of learning experiences: (1) Coursework, (2) Observation, (3) Practice, and (4) Feedback. 


Specifically, the survey items asked teachers to rate on a five-point scale (from 1 = “Rarely/Never” 
experienced to 5 = “Very often” experienced) the extent to which they (1) read about, heard about, or saw 
a role play of the strategy (such as during coursework); (2) observed a teacher using the strategy in a K— 
12 classroom; (3) practiced the strategy in a K-12 classroom; or (4) received feedback on their use of the 
strategy. (See figure ES.2.) This taxonomy of types of learning opportunities aligns with qualitative 
research and theory that suggest that the four types form a continuum and that all are required for learning 
to take place (Grossman, Compton et al. 2009; McDonald, Kazemi, and Kavanagh 2013). Within each 
competency area and within each type of learning opportunity, teachers’ ratings were averaged and 
reported as very few (1 < avg < 2); few (2 < avg < 3); some (3 < avg < 4); or many (4 < avg <5). 


3 Bill and Melinda Gates Foundation. Measures of Effective Teaching: 1 - Study Information. ICPSR34771-v2. 
Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2013-09-23. 
https://doi.org/10.3886/ICPSR34771.v2. 
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Figure ES.2: Example Survey Items for “Maintaining a Positive Classroom Climate” Competency 
Area 


Communicating respect and warmth both in words and 
non-verbally (making eye contact, being in physical 
proximity) when interacting with individual students and 
the class as a whole. 

Demonstrating knowledge of and interest in students’ 
lives inside and outside of school. 

Listening closely and with genuine interest when students 
talk and encouraging students to listen to each other. 
Encouraging cooperation among students, such as 
working together and sharing materials. 

Responding to students’ efforts and participation with 
positive comments (e.g., “What a great idea!,” “You guys 
are working together really well on that project.”). 


SOURCE: Study’s teacher survey, 2015. 


Given that the competency areas included on the survey cover aspects of teaching previously shown to be 
related to student outcomes, it is worthwhile to see whether those experiences were included in teachers’ 
preparation and to what extent. In addition, recent emphasis on increasing teachers’ clinical experiences 
leads to interest in the extent to which preparation included hands-on experiences with feedback. Finally, 
the relationship between preparation experiences and early teaching effectiveness might generate 
hypotheses for improving teacher preparation. 


Key Findings 
Average Frequencies and Variation of Preparation Experiences 


Teachers’ preparation experiences included a broad array of competency areas, 
although teachers reported more frequent experiences in some competency 
areas than others. 


The study examined the extent to which teachers reported experiences across the 13 competency areas in 
order to explore whether programs include the broad array of content believed to be important to 
preparing effective teachers. At least 64 percent of teachers reported some or many preparation 
experiences with teaching strategies in 12 of the 13 competency areas. Effective Instruction for English 
Learners was the one competency area in which less than half of teachers (43 percent) reported at least 
some experiences (figure ES.3). 


Figure ES.3: Teachers’ Preparation Experiences by Competency Area 
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NOTE: Teachers’ responses to the survey questions about preparation experiences ranged from 1 to 5, where 1 = “Rarely/Never” 
and 5 = “Very often.” This figure shows the proportion of teachers in four groups based on teachers’ average preparation 
experiences with each competency area. The 13 competency areas are listed in order of highest to lowest average score. Sample 
size varied between 3,249 and 3,289 due to nonresponse. All sample averages are statistically different from one another (p < .05) 
except these: (1) Effective English Language Arts Instruction, Building Students’ Higher-Order Thinking Skills, and Facilitating 
Extended Classroom Discussions are not different from one another; and (2) Effective Mathematics Instructions and Building 
Comprehension of Academic Concepts are not different from each other. 

FIGURE READS: Three percent of teachers reported having had very few preparation experiences with Maintaining a Positive 
Classroom Climate. The sample average rating for preparation experiences with Maintaining a Positive Classroom Climate was 4.0. 
SOURCE: Study’s teacher survey data, 2015. 


Although teachers learned about competency areas through all four types of 
learning opportunities, Coursework was the most frequent and Feedback was 
the least frequent. 


Teachers reported that they received preparation experiences most frequently through Coursework 
(average of 3.8 on the five-point scale), and least frequently through Feedback (average of 3.4). This 
pattern is also reflected in the proportion of teachers who reported at least some preparation experiences 
through each type of learning opportunity: 83 percent for Coursework versus 65 percent for Feedback 
(figure ES.4). However, nearly a quarter or more of teachers reported few or very few experiences with 
three of the four types of learning opportunities: 24 percent for Observation, 26 percent for Practice, and 
34 percent for Feedback. 
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Figure ES.4: Teacher-Reported Frequency of Preparation Experiences by Learning Opportunity 
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NOTE: Teachers’ responses to the survey questions about preparation experiences ranged from 1 to 5, where 1 = “Rarely/Never” 
and 5 = “Very often.” This figure shows the proportion of teachers in four groups based on teachers’ frequency of preparation 
experiences (averaged across the 13 competency areas) obtained through each type of learning opportunity. The four types of 
learning opportunities are listed in order of highest to lowest average score. Sample size varied between 3,286 and 3,288. All 
sample averages are statistically different from one another (p < .05). 

FIGURE READS: The average frequency of preparation experiences obtained through Coursework was very few for 3 percent of 
teachers. The sample average rating for preparation experiences obtained through Coursework was 3.8. 

SOURCE: Study’s teacher survey data, 2015. 


Within each type of learning opportunity, teachers reported more preparation 
experiences with strategies for creating a productive learning environment than 
with strategies for promoting analytic thinking skills. 


Using a statistical technique called factor analysis, the study categorized each of the 13 competency areas 
into two broad categories of preparation experiences--creating a productive learning environment and 
promoting analytic thinking skills. Regardless of type of learning opportunity examined, teachers reported 
more experiences with creating a productive learning environment than with promoting analytic thinking 
skills. For example, for Coursework, the average rating for strategies for creating a productive learning 
environment was 3.8, compared to the average rating of 3.6 for strategies for promoting analytic thinking 
skills (not shown). 


Relationships Between Preparation Experiences and Teaching Effectiveness 


Teachers who reported more frequent preparation experiences with strategies 
for creating a productive learning environment were more effective in the 
classroom than were teachers who received fewer such experiences. 


For ELA, the relationships between preparation experiences for creating a productive learning 
environment and teaching effectiveness were positive (i.e., more frequent preparation experiences were 
related to greater effectiveness) and statistically significant for three of the four types of learning 
opportunities: Observation, Practice, and Feedback. For math, the relationships between preparation 
experiences for creating a productive learning environment and teaching effectiveness were positive for 
all four types of learning opportunities and statistically significant for Practice (figure ES.5). 
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Figure ES.5: Relationships Between Preparation Experiences and Teaching Effectiveness in ELA 
and Math 
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* Coefficient for relationship of experience to effectiveness is significantly different from zero, with p < .05. 

** Coefficient for relationship of experience to effectiveness is significantly different from zero, with p < .01. 

NOTE: Coefficients for the relationship of preparation experiences to ELA teaching effectiveness were estimated using data from 

n = 2,032 ELA teachers. Coefficients for the relationship of preparation experiences to math teaching effectiveness were estimated 
using data from n = 1,894 math teachers. 

FIGURE READS: The horizontal line represents average teaching effectiveness for teachers in the sample. Bars above the line 
indicate that the preparation experience is related to greater effectiveness, while bars below the line indicate that the preparation 
experience is related to lesser effectiveness. The regression coefficient for the relationship of preparation experiences with creating 
a productive learning environment obtained through Coursework to ELA teaching effectiveness is 0.045, meaning a one-unit 
increase in preparation experiences is related to an increase of 0.045 standard deviation units above average effectiveness. This 
estimate is not statistically different from zero. 

SOURCE: Study’s teacher survey data, 2015. Effectiveness measures: study’s teacher-student linked data, 2012-15. 


To give a sense of the magnitudes of the statistically significant relationships between preparation 
experiences and teaching effectiveness, we compared the sizes of those relationships to the sizes of the 
average differences in teaching effectiveness between first- and second-year teachers in ELA and math. 
The difference in ELA effectiveness that was associated with a one-point difference in preparation 
experiences for creating a productive learning environment through Observation, Practice, or Feedback 
was about half the size of the average difference in ELA effectiveness between first-year and second-year 
teachers. The association between preparation experiences for creating a productive learning environment 
through Feedback and math effectiveness was about one fifth of the size of the difference between first- 
and second-year teachers in math effectiveness. 


Teacher preparation experiences with strategies for promoting analytic thinking 
skills were not related to teachers’ effectiveness in the classroom. 


The relationships between preparation experiences for promoting analytic thinking skills and teaching 
effectiveness were not statistically significant for either ELA or math. This finding was consistent across 
all four of the types of learning opportunities (Coursework, Observation, Practice, Feedback). 
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Concluding Thoughts 


There are some limitations to this study, which provides information about the preparation experiences 
reported by a large, but purposive sample of elementary-level teachers in general-education classrooms. 
The study examined effectiveness only for teachers in upper elementary grades for which test scores were 
available. In addition, preparation experiences were self-reported by teachers and may have been subject 
to measurement error. Despite these limitations, the detailed nature of these data provides more 
information about teacher preparation than typically provided in the literature. 


The study found variation across teachers in the extent of the preparation experiences they reported both 
for competency areas and types of learning opportunities. This variation allowed the study to explore 
relationships between preparation experiences and classroom effectiveness during teachers’ first few 
years of teaching. These types of analyses are a useful first step in generating hypotheses for improving 
teacher preparation, a primary objective of this study. 


The study found significant relationships between strategies for creating a productive learning 
environment and teaching effectiveness. Practice was the only type of learning opportunity that showed a 
statistically significant relationship to teaching effectiveness in both ELA and math. Having obtained 
preparation experiences through Coursework was not significantly related to effectiveness for either 
subject area. These analyses suggest that there might be promise in preparation programs emphasizing 
strategies for creating a productive learning environment, particularly using more hands-on methods for 
learning these strategies. This finding is not causal and more rigorous research is needed to reinforce it. 


Regardless of the type of learning opportunity, there were no statistically significant relationships 
between preparation experiences with promoting analytic thinking skills and teaching effectiveness. The 
lack of relationships is concerning, since research suggests that students’ ability to employ higher-order 
thinking skills about academic material is a predictor of overall academic success (Allen et al., 2011; 
Hamre and Pianta, 2005). However, it would be premature to conclude that experiences focused on 
promoting analytic thinking skills are not an important component of effective preparation programs. It 
could be that the particular strategies within promoting analytic thinking skills that this study focused on 
may not be the most essential ones, and that other strategies not measured but aimed at improving 
students’ analytic thinking may be effective. More exploratory research is needed to better understand this 
relationship. 
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